Separating the Wheat from the Chaff in Internet-Mediated User Feedback

The Internet enables cheap, rapid, and large-scale distribution of software for evaluation purposes. It also presents hitherto unprecedented, and currently underutilized, opportunities for increasing user-developer communication in software development. For instance, the Internet can be used as a medium for collecting “direct” user feedback in the form of subjective user reports, as well as “indirect” feedback in the form of automatically-captured data about application and user behavior. Both of these practices, however, face a number of challenges that can be summarized in the following statement: there is more feedback to be collected — ranging in quality from useful to useless — than there is time and resources to sift through and act upon the meaningful parts. This paper describes an Internet-based approach for capturing user feedback — both “direct” and “indirect” — that attempts to address this problem by focusing feedback collection based on the notion of “usage expectations” in the development process.